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question: In secondary mathematics, what are the general and 
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strategies on learning in cooperative groups? Two instructional 
programs for cooperative learning were compared: (1) an experimental 

program with training in the use of social and cognitive strategies, 
and (2) a control program similar in mathematical content and overall 
instructional design but lacking instruction in use of social and 
cognitive strategies. Teaching cognitive and social strategies had an 
expected, positive effect on the mean Mathematical Reasoning Ability 
test and on the domain-specific test on measurement and measures. 
Intervention had no effect on the mean achievement of the 
domain-specific test for information processing. Differential effects 
between high- and low-achieving students were found on both the 
Mathematical Reasoning Ability test and the Information Processing 
test, but none on the Measurements and Measures test. Low-achieving 
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Abstract 

This paper reports on an intervention study into the effects of a program on student 
achievement in secondary mathematics education. The program involves instructions in 
social as well as cognitive strategies. In the development of the instructional programs 
special attention was given to low-achieving students in mathematics. Various studies show 
that training students in the use of social and cognitive strategies contributes to learning. 
However, low-achieving student seem practically unable to benefit from such interventions. 
The experiment addressed the following research question: in secondary mathematics, what 
are the general and differential effects of training social and cognitive strategies on learning 
results in cooperative groups? 

The answer to this question involved the design of an intervention study. In the present 
study two instructional programs for cooperative learning in mathematics are compared: (1) 
an experimental program with training in the use of social and cognitive strategies, and (2) a 
control program without instructions in the use social and cognitive strategies. The programs 
are similar in respect of mathematical content and overall instructional design. 

Three hypotheses were formulated (i) The program hypothesis : The program has a 
positive effect on learning outcomes, (ii) The differential effect hypothesis: High- and 
low-achieving students will benefit differently from cooperative learning and (iii) The 
remedial instruction hypothesis: Low-achievers in the experimental program will benefit 
more from working in small groups than low achievers in the control program as a result of 
the special remedial instruction in the experimental program. 

The research was conducted in two schools for secondary education, comprising 18 
classes with a total of 450 students. The design was a pretest-posttest control group design. 
The data were analyzed from a multi-level perspective. The outcomes show a number of 
effects of the intervention. However not all hypotheses were confirmed. 

Analyses of variance and multi-level analyses were performed: (1) on Mathematical 
Reasoning Ability as measured by a standardized test; and (2) on two different experimenter- 
designed, domain- specific tests in mathematical achievement. Teaching cognitive and social 
strategies has an expected, positive effect on the mean Mathematical Reasoning Ability and 
on the domain specific test on 'Measurement and Measures ’. The effect size regarding the 
Mathematical Reasoning Ability test is 0.2, which is a small effect. The effect sizes of the 
domain-specific 'Measurement and Measures’ is 0.50, a medium effect. No effect of the 
intervention was found on the mean achievement of the domain specific test 'Information 
Processing’ (Cohen, 1988). 

In addition, differential effects were found both on the 'Mathematical Reasoning Ability ’ 
test and on the ' Information Processing ’ test, but none on the ' Measurements and 
Measures’ test. The differential effects on the standardized Mathematical Reasoning Ability 
test relate to the high-achieving students, while the differential effects on the Information 
Processing test relate to the low-achieving students. However, the direction of these effects 
was different per criterion measure. Low-achieving students in the experimental condition 
gain more on the domain specific test than low-achieving students in the control program. 
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Introduction 

The past twenty years have shown a revival of interest in cooperative learning in many 
countries. This interest grew because cooperative learning programs promised success for all 
students. Students working in cooperative settings outperform their counterparts working in a 
more competitive setting (Qin, Johnson & Johnson, 1991). In addition to these findings Webb 
and Farivar (1994) found evidence that low and high achieving students benefit differently 
from cooperative groupwork. These difference in learning gains between low- and high- 
achieving students are related to social and cognitive factors. Terwel and Van den Eeden 
(1992a, 1992b) showed that low- and high-achieving students benefit differently from 
cooperative group work. Hoek, Van den Eeden and Terwel (1996, submitted) show that 
special instruction can mitigate ‘the stronger are getting stronger’ factor. Their training 
program provided more students with ‘access to resources', allowing them to participate 
effectively in cooperative problem solving It was shown that the low-achieving students 
profited more from the intervention programs than the low-achieving students in the control 
program. From these conclusions certain ideas emerged for the design of a new experiment, 
in which instruction combined the use of social and cognitive strategies (Dahr & Resh, 1994; 
Hoek, Terwel & Van den Eeden, 1996; Rosenshine, Meister, and Chapman, 1996; 
submitted). 



Theory 

According to Schoenfeld (1985) students have difficulties in problem solving because they 
have certain belief systems. In his view students have 3 basic beliefs about mathematics: 

(1) Mathematics has almost nothing to do with real thinking or problem solving; 

(2) Mathematical problems should either be solvable within 10 minutes, or you stop working 
on the problem; 

(3) Only geniuses can do mathematics. If I forget something, no problem. I am not a genius 
and take procedures at face value. I do not try to understand why such procedures work 
(Schoenfeld, 1985). 

The construction of these belief systems are to a certain extent the result of the instruction the 
students receive from their teacher. It is therefore not surprising that students do not develop 
meta-cognitive skills as long as virtually all instruction focuses on declarative and procedural 
knowledge and the students only have to listen to, watch and imitate the teacher (Greeno, 
1991). This ‘information transmission model’ causes the absence of any notion of being one’s 
own coach, let alone being the coach of one’s classmates. Most teachers show the problem- 
solving procedure without any explanations as to why such procedures should work. 
According to Schoenfeld the teacher can bring the procedure into the open by asking students 
questions of the following kind: ‘What are the options in trying to solve this problem? ‘How 
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should we proceed?’ These strategic questions are asked to control and to guide the individual 
problem-solving processes and to give the students the opportunity to understand why, when 
and how a problem-solving procedure is used. In line with this type of reasoning Hattie, et al., 
conclude in their meta-analysis that training in problem solving is only effective if it becomes 
clear to the students why, when and how a procedure is used and why similar questions help 
students understand the procedure (Hattie, Biggs and Purdie, 1996). However, students, and 
even adults, have problems asking themselves these questions because a great deal of human 
behavior is largely automatic, and often unconscious of the situational representation. 

Schoenfeld (1994) shows that students have difficulties in taking the contextual 
information into account, especially if they are required to answer problems in rich contexts, 
for example, in providing solutions to problems of the type: calculate the number of 36-seater 
buses needed to transport 100 persons. Most students would provide the answer 2 remainder 
28; that is, without taking the wider context into account. Students were using the right 
procedure to calculate the answer, while ignoring the correct strategy for a final answer. In 
other words, Schoenfeld’s students did not reflect on their answer. The example shows that it 
is important to reflect on solutions. But reflecting is not the only important aspect of problem 
solving. According to Pressley & McCormick “All successful problem solving programs 
encouraged students to reflect and check their work; in particular, students were encouraged 
to (1) check their answers; (2) compare the answers they obtained with the estimated answer; 
(3) solve the problem in another way; (4) summarize how they solved the problem; and (5) 
construct a problem that requires the same solution strategy” (Pressley & McCormick, 1995, 
page 402). 

From a pure cognitive perspective the problem-solving process contains certain 
components, these are represented in Figure 1. 



Insert figure 1 here 



Figure 1 represents the knowledge used during the (mathematical) problem-solving process. 
The task description contains elements that are relevant and irrelevant to the solutions of a 
problem (situational knowledge). To solve a mathematical problem it is necessary to use a 
procedure. In order to be able to use a procedure the student has to know certain facts or 
formulas (declarative knowledge). This process is coached or influenced by the strategic 
knowledge of the individual. Students can ask themselves questions such as: What is it I have 
to calculate? Why I am doing this? Did I give the right answer? 

This description suggests that the components in question consist of discrete, successive 
operations. But it should be kept in mind that such processes operate in parallel, especially 
when experts are doing the solving (Chi, Feltovich and Glaser, 1981). 

Low-achieving students in particular encounter difficulties while problem solving. The 
main reason why they do not use the right strategy is that they have deficiencies in both 
declarative and procedural knowledge (Chi, Feltovich & Glaser, 1981). Various researchers 
are convinced that (mathematical) problem solving becomes easier if the student has a well- 
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organized domain-specific, declarative knowledge base and knows how to use domain- 
specific procedures (Schoenfeld, 1987; Bransford et al, 1989; Bonner, 1990; Resnick en 
Klopfer, 1989). In a review of the literature Nelissen (1995) concludes that, besides 
declarative knowledge and procedural knowledge, reflective (strategic and meta-cognitive) 
knowledge is required to solve mathematical problems (see also Verschaffel, 1995). 

In the use of strategic knowledge the student can use three types of resources: the 
teacher, classmates and curriculum materials. 

The teacher may be used as a model for problem solving. However, it should be kept in 
mind that, during the modeling of the problem solving process, students are not passive 
receivers of information, but have to actively construct their knowledge, and, furthermore, 
that learning is influenced by the physical and social environment (Steffe and Gale, 1995). 

According to Dar and Resh (1994) students may use their classmates as resources. 
Cooperative learning setting turns out to be suitable as a context for this strategy. 

For several reasons Schoenfeld (1983, 1989a) advocates group work to teach students to 
use problem-solving strategies. The teacher has the opportunity to coach the students while 
they are solving problems together. Working together in small groups implies discussion, 
which encourages the development of (meta-) cognitive strategies. Groupwork teaches the 
students to practice collaboration in real settings, and externalizes as well as differentiates the 
roles and processes involved in problem solution. Within such small group settings the 
students realize that their fellow group members may also have to struggle with the problems 
given to them. This does not mean that placing students into small groups is enough, since 
groups are only effective under appropriate conditions (Salomon and Globerson, 1989; Good, 
Mulryan & McCaslin, 1992). Various studies have shown that students with different abilities, 
working in small groups, do benefit differently (Webb and Farivar, 1994, Van den Eeden and 
Terwel, 1994). These differences can be explained on the basis of sociological theories and in 
terms of a cognitive perspective. Both perspectives have consequences for the students' 
training in the use of problem-solving strategies in a cooperative setting. We now turn to a 
description of these perspectives. 



Two perspectives on the differential effects 

At least two theoretical orientations can be maintained in explaining differential effects: a 
sociological orientation and a cognitive one. These have one main concept in common, 
namely, ‘access to resources’ (Cohen and Lotan, 1995; Prawat, 1989). From a sociological 
theoretical perspective it is conceivable that students within heterogeneous groups have 
different access to resources. Students in groups develop status orders, which are based on 
perceived differences in academic success. Status differences have the effect of depressing 
the participation of low status students in small group interaction. As a consequence, 
differences in achievement increase (Cohen and Lotan, 1995). However, it is not only the rate 
but also the nature of the participation that influences learning in cooperative groups. 

Students learn more from receiving elaborated help from other group-members and less from 
receiving low-level elaboration (for example, from receiving only the answer to a problem). 
From the sociological perspective learning can be promoted by enhancing the nature of 
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participation. This can be achieved by social strategy training and by influencing classroom 
interaction patterns in a direction which guarantees all students ‘equal access to resources’; 
e.g. by promoting and improving helping behavior in cooperative groups (Webb, 1989, 1991, 
1995; Cohen and Lotan, 1995). 

From a cognitive perspective, low-achieving students benefit less from group work since 
they lack prior knowledge (declarative, procedural, strategic). Differences in access to 
resources are primarily seen in terms of knowledge and strategies. Low-achieving students 
are not always able to use knowledge and problem solving strategies (at the right moment). 

As a consequence of deficiencies in strategic and metacognitive knowledge low-achievers are 
not always able to cope with the strategies used by high-achievers. On their part, 
high-achievers are not always able to explain their own routine strategies. From the cognitive 
perspective the promotion of learning in cooperative groups can be realized by improving 
strategy awareness. This can be done by training students in the use of (meta-) cognitive 
strategies, e.g., problem solving strategies and control strategies such as planning, 
monitoring, checking, and revising. This kind of training is most successful if conducted 
within a school subject or domain in the context of the school curriculum, rather than in 
isolation. Highlighting reflection processes is very important (Ausubel, 1968; Prawat, 1989; 
Resnick, 1989; De Corte and Verschaffel, 1988; Schoenfeld, 1987, 1992; Chinnappan and 
Lawson, 1996). 

The sociological and the cognitive perspective are complementary in explaining the 
differential effects of small-group learning. Various social and cognitive factors may prevent 
low-achievers from obtaining access to resources and, consequently, may prevent learning. 

As a result, differences between high and low-achieving students may increase. 

What do we learn from the research literature into the effects of social or cognitive 
strategy training? In general there is reason to expect positive effects from such training. 

There is some research evidence that the effects, in most cases, are positive. Webb and 
Farivar (1994) reported positive effects of small-group social interaction training for some 
groups but not for others. There is additional evidence from research into cognitive strategy 
training. In a recent meta-analysis, including 5 1 studies, positive overall effects were found, 
especially for the type of strategy training in which reflection on the how, when, where and 
why of strategies was stimulated. However, researchers report that low- achieving students 
are unable to benefit from interventions of most kinds. Surprisingly, they also found a few 
exceptions (Hattie, Biggs and Purdie (1996). Lastly, in an experimental study into the effects 
of problem solving training in mathematics Chinnappan and Lawson (1996) also found 
positive effects on performance of both high- and low-achieving students. 

It may be concluded that there is some evidence regarding the effectiveness of training in 
the use of social or cognitive strategies. However, there is good reason to examine the 
problem of differential effects for high- and low achieving students in greater detail, 
especially in the context of cooperative group learning. Given the concerns regarding low- 
achieving students mentioned above, special attention will be given in the present study to the 
counterbalancing forces of remedial instruction and guidance ‘scaffolding’ for the low- 
achieving students. The concept of scaffolding originates from sociocultural ('Vygotskyan') 
theories. Scaffolding is closely related to one of the mainstays of sociocultural theory called 
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‘the zone of proximal development’. The metaphor of a scaffold resembles the finely tuned, 
temporary support that can be removed when no longer needed. (Azmitia and Perlmutter, 
1989; Brown and Palinscar, 1989; Collins, Brown and Newman, 1989; Tudge, 1990; 
Rosenshine, Meister and Chapman, 1996). 



Elements of a successful training 

Our aim is to teach students to use strategies and to reflect on their given answers in 
cooperative settings. This means that students have to be trained in the use of both social 
strategies and cognitive strategies. What kind of elements should such training contain? 

According to Schoenfeld it should contain scaffolding of instruction and coaching. A 
good teacher coaches the students during their problem-solving attempts, by posing questions 
such as: 

• What (exactly) are you doing? Can you describe it precisely? 

• Why are you doing it? How does it fit in with your solution? 

• How does it help you? What will you do with the outcomes when you obtain it? 
(Schoenfeld, 1992, p. 356). 

In their meta-analysis, Hattie, Biggs and Purdie (1996), conclude that the best results are 
obtained when strategy training is meta-cognitive, with appropriate motivational and 
contextual support. This means that the students have to learn and to understand the strategy 
used. Hattie et al. recommend that training programs (a) be designed in context; (b) use tasks 
with the same content as the target domain; (c) promote a high degree of learner activities and 
meta-cognitive awareness. It should become clear to the students what the strategy is and 
when it is appropriate to use certain domain-specific procedures and why these are used. The 
results show that skill-training packages are not as effective as metacognitive and contextual 
interventions (p. 129). 

In line with the conclusions of Hattie, Biggs and Purdie (1996) Van Streun (1989) 
concludes: “Explicit attention for heuristic methods and gradual and limited formulating of 
mathematical concepts and techniques in mathematical education achieve a higher problem 
solving ability than implicit attention for heuristic methods and late and little formulating of 
mathematical concepts and techniques”. He also concludes: “Heuristic methods have to be 
integrated in cognitive schema’s of mathematical concepts and techniques and applications” 
(Van Streun, 1989, p. 109). 

Research from the sociological perspective shows that all students in cooperative settings 
should have the possibility of participating in group interaction. Attention should be paid to 
explanations given by the (high-achieving) students, and to status differences (Cohen, 1994; 
Webb, 1989). 

We have tried to develop an intervention program, taking into account some theoretical 
and experimental results provided in the literature. The program contained training both in the 
use of problem-solving strategies and in the use of strategies for effective cooperative small 
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group work. The experimental program was designed to test a number of hypotheses, which 
are discussed immediately below. 



Research question and hypotheses 

The experiment addresses the following research question: in secondary mathematics, what 
are the general and differential effects of training social and cognitive strategies on learning 
results in cooperative groups? Three hypotheses were formulated. 



The program hypothesis 

The experimental program (training in the use of social strategies and cognitive strategies, 
and remediation) has a positive effect on learning outcomes (Qin, Johnson and Johnson, 
1991; Webb and Farivar, 1994; Hattie, Biggs and Purdie, 1996; Chinnappan and Lawson, 
1996). 



The differential effect hypothesis 

High- and low-achieving students will benefit differently from cooperative learning. In 
operational terms: as a consequence of their superior resources high-achieving students will 
gain more per score unit on the pretest than low-achieving students. Thus a curvilinear 
relation between pretest and posttest is expected in the experimental groups as well as in the 
control group because both conditions contain cooperative learning. 



The remedial instruction hypothesis 

Given a context of experimental programs in which special (remedial) instruction and 
guidance is given to the lower-achieving students by the teacher, it is expected that 
low-achievers in the experimental program will benefit more from working in small groups 
than low achievers in the control program (also consider the concept of ‘scaffolding’ in socio- 
cultural theory: Azmitia and Perlmutter, 1989; Brown and Palinscar, 1989; Collins, Brown 
and Newman, 1989; Rosenshine, Meister and Chapman, 1996). 

These hypotheses were tested using a multilevel model. In this model the direct effect of the 
pretest score on the posttest score is controlled. In the analyses no dichotomization between 
high- and low-achieving students is realized but the student sample remains intact through the 
use of continuous variables. 



Method 

Participants 

The data were obtained in the spring of 1996. The participants were 444 students from 18 
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classes (213 females and 237 males) in two Dutch secondary schools, with heterogeneous 
classes. Within both schools the experimental as well as the control programs were 
implemented. The teachers in the experiment were randomly divided over both programs. 
When a teacher had more classes to teach, all these classes were put in the same program. In 
all 222 students from 9 classes participated in the experimental and control programs. In 
order to eliminate possible school effects, both programs were implemented at both schools. 



Design 

The research made use of an quasi- experimental pretest-posttest control group design, in 
which the effects of special instruction for group work was compared to effects of groupwork 
without special instruction. The research lasted for approximately 1 0 weeks in the first year 
of secondary education. Both the experimental and control programs covered 2 series of 14 
lessons in mathematics in the first year of secondary school (12-13 year olds). The first series 
of lessons (‘Measurements and Measures’) contained problems about measurement units, 
circumference and area. The second series of lessons (‘Information Processing’), contained 
problems about tables and figures with different kinds of information, maps and graph 
representations. The design of this experiment is shown in Figure 2. 



Insert figure 2 here 



The arrows in Figure 2 marked *1* represent the direct effect of the Mathematical 
Reasoning Ability pretest on the Mathematical Reasoning Ability posttest; the arrows marked 
‘2’ represent the direct effect of the domain specific mathematical pretest on the domain 
specific mathematical posttest. The arrows marked ‘3’ represent the program effect on the 
posttest. 



Assessments 

A Mathematical Reasoning Ability test was administered to all participating students prior to, 
and at the end of the experiment. The learning-effect (testing-effect) for students doing the 
same test before and after the implementation of the programs was assumed to be identical 
for both programs. The test consisted of two sub-scales, numbers 3 and 4 of the Priifsystem 
fur School- und Bildungsberatung (PSB), (Horn, 1969). According to Horn (1969) both 
scales have a high load on the factor ‘reasoning’. In earlier research correlations between PSB 
sub-scales 3 and 4 and mathematical achievement between .50 and .80 were found (Herfs, 
Mertens, Perrenet and Terwel, 1991; Aurin, 1966). Scores on the pretest were used as control 
variable, with the posttest scores as dependent variable. The total test contained 80 items, 
with possible score ranges from 0 to 80 correct items. The alpha reliability-coefficients of the 
pretest and the posttest are both a = 0.80. 
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Besides the Mathematical Reasoning Ability test we administrated two domain- specific 
pretests and posttests for each series of lesson. All four tests, two pretests and two posttests, 
were experimental, with open end problems. Scores on the pretest were used as a covariant 
with the posttest as dependent variable. The reliability (a) of the experimental tests are given 
in Table 1. 



Insert table 1 here 



The reliability of the tests in this experiment varies between a= 0.60 and a= 0.83. 



Procedure and Materials 

The experiment started with a teacher training program. In this training the basic instruction 
(AGO) model used in the experiment was explained to all the participating teachers. 

The AGO model was used as a point of departure in the design of the instructional programs. 
This instructional model combines aspects of cooperative learning and adaptive instruction. 
AGO is a Dutch acronym for ‘adaptive instruction and cooperative learning’. The model is 
based on theories about cooperative learning and cognitive learning theory. The AGO model 
is designed for the middle grades, 12 to 16 year-old students (Freudenthal, 1973; Terwel, 
1990; Terwel, Herfs, Mertens and Perrenet, 1994). In terms of Mason and Good' s 
classification (1993), we may describe the model as a whole-class model that allows for 
student diversity through small-group ad hoc remediation and enrichment on a daily basis. 
The AGO-model consists of the following stages: 

1 . Whole-class introduction of a mathematics topic in real-life contexts; 

2. Small-group cooperation in heterogeneous groups of four students; 

3. Teacher assessments: diagnostic test and observations; 

4. Alternative learning tracks depending on assessments. These tracks consist of two 
different modes of activity: 

4a. Individual work at own pace and level (enrichment) in heterogeneous groups with the 
possibility of consulting other students. 

4b. Opportunity to work in a remedial group (scaffolding) under direct guidance and super- 
vision of the teacher; 

5. Individual work at own level in heterogeneous groups with possibilities for students to 
help each other; 

6. Whole-class reflection and evaluation of the topic; 

7. Final test. 

The AGO model was the basic instructional design for the experimental program and the 
control program; however some modifications were made, especially in stages 4 and 5. 
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In the present project heterogeneity was abandoned, particularly with reference to the 
cognitive program stages 4 and 5, and replaced by homogeneous small groups. This 
modification was advocated by the teachers who, for practical reasons, preferred a kind of 
within-class ability grouping (within-class setting with low- medium- and high-achieving 
students) in stages 4 and 5. Rearrangement of students was thus no longer required during 
these stages. In addition, low-achieving students can be located more easily by the teacher 
because they are seated in the same small ability-group. 

In the control program the grouping practice was the same. However, there was no 
special (remedial) instruction or guidance for students in the low-achieving group. 

The teachers of the experimental program received an extra training session to practice 
the training they had to give to their students. During this training session the coaching for 
mathematical problem solving as well as small groupwork coaching were explained and 
practiced. 

The experimental program started with a student training program. During the first part 
of this training with whole-class instruction the teacher demonstrates the problem-solving 
model to be used during this experimental period (Van Streun, 1989, 1994). At this stage the 
teacher serves as a leader or expert for his students in a cognitive apprenticeship model. The 
methods used involved exploration, coaching, articulation, and reflection. (Collins, Brown 
and Newman, 1989; Schoenfeld, 1985, 1992). 

In the second part of the program the teacher trains the student in cooperative group work 
by using the ‘Broken Circles Problem’ and the ‘Master Designer Problem’ (Cohen, 1986, 
page, pages 159 to 164). 

For the teacher training two manuals were developed. The first of these, designed for all 
teachers, contained the following elements: (1) the design of the experiment, with the 
measurements; (2) the content of the domain specific subjects. The second, more specific, 
manual for the teachers of the experimental program contained: (1) the problem solving 
model and the small group coaching used in the experimental program; (2) exercises for the 
student training program. 

Two series of lessons were developed for the experiment: (1) Measurement and 
Measures and (2) Gathering Information. The mathematical content of both the control and 
experimental programs was the same. However, the experimental booklets contained extra 
assignments for problem solving and small groups cooperation. 
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Data and analysis at the individual level 

Table 2 shows the results of the students in the control and experimental program. 



Insert table 2 here 



Analysis of variance on the posttest score corrected for the pretest score shows a significant 
effect on the Reasoning Ability test (p<0.05) and on the test for Measurements and Measures 
(p<0.05), but not on the Gathering Information posttest (p>0.05). This indicates that the mean 
achievement of the students in the experimental program is higher than the mean achievement 
of the students in the control program on the Measurements and Measures posttest, but not on 
the Gathering Processing posttest. 

Analysis variance on the posttest Mathematical Reasoning Ability corrected for the score 
pretest shows a significant difference in favor of the experimental program (p<0.05); the 
same holds for the score on the Measurements and Measures posttest, corrected for the pretest 
(p<0.05). The analysis of variance on the Gathering Processing-2 posttest reveals no 
significant differences between the experimental and control programs (p>0.05). 

The effect sizes of the domain- specific Measurement and Measures, and Information 
Processing tests are respectively 0.50, a medium effect, and 0.00, no effect. The effect size of 
the Mathematical Reasoning Ability test is 0.2, a small effect (Cohen, 1988). 

The learning gain on the Mathematical Reasoning Ability test for the control group is 0.4 
and for the experimental group 1.85. On the domain-specific tests it is not possible to 
calculate the learning gain because the total obtainable score on the pretest and posttest per 
domain-specific test differ. 



Insert table 3 here 



The slopes represented in Table 3 of the different programs on the Mathematical Reasoning 
Ability test differ only slightly, while the intercept of the experimental program is higher than 
the intercept of the control program. The intercepts of the different programs on the 
Measurements and Measures test do not differ significantly, while the slopes are different. 
The intercept and the slope of the experimental and control program differ significantly on 
the Gathering Information test. It may be concluded that the low-achieving students profited 
more from the experimental program than the high-achieving students. Are these preliminary 
conclusions confirmed by a multi-level analysis? 




12 



13 



Results of the multi-level analysis 

To assess the effects of the programs regarding varying degrees of achievements, the 
appropriate tool to use is a multilevel model of analysis (Terwel & Van den Eeden, 

1992a, 1992b; Van den Eeden & Terwel, 1994). The ML3-E statistical program was 
used in the analysis (Prosser, Rasbash & Goldstein, 1993). A multilevel analysis 
with a continuous variable as indicator of high- and low-achieving students is 
superior to a regression analysis or multilevel analysis in which separate categories 
for low-, mediate- and high-achieving students are used. An analysis with continuous 
variables is more efficient and the resulting estimates are more accurate. 

The multilevel analysis was directed at the relation between achievement after 
completion of the program (the posttest) and at variables on the individual student level 
(pretest) and the class level (program). In the model of analysis these relations are described 
in terms of regressions of a dependent variable on the independent variables at student and 
class levels. 

The model basically consists of two steps: the first step concerns a within-group 
regression, in the second step the results of the first step are introduced in a between-group 
regression. The two steps of the model can be formulated as follows, with the classroom 
taken as group level. The student-level regression of the first step is expressed in the 
following equation: 

Posttesty = J3 0 j + py 

where 
i: 

j- 

m: 

Pmf 
Pof- 

e y" 

Posttesty: 

Pretesty" 

Ability-Difference 



Pretesty + P 2 j Ability-Differencey + eu 



individual student (i = 1 5 1 1) 

class (j = 1,...,21) 

independent variable at student level (m = 1,2) 
slope of regression of post-test on variable m of class j 
intercept of class j 

disturbance term, with mean 0 and variance s 
score on mathematical reasoning test after completing the program 
score on mathematical reasoning test at the beginning of the program 
product of mathematical reasoning ability- 1 and the test at the beginning 
of the program (Mathematical Reasoning Ability, Measurements and 
Measures, and Information Processing) 



The second concerning the regressions between classes of the slopes b m j on the class 
variable (in casu the program variable). 

Pmj = Ymo + Y m\ Program / + u m j 



where 



y 00 : intercept of regression of p o j on class variable Program 
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Ymn- slope of regression of p m j on class variable Program (n = 1) 

Umj' disturbance term of the regression of p m j, with mean 0 and variance s 2 m 

The model contains the between-class variance, to explain occasional differences in 
intercepts and slopes. The part of the model that contains the slope coefficients of 
specific variables is the ‘fixed part’ of the model; the part that contains the 
disturbance terms being the ‘random part’. 

Figure 3 represents the basic model of the multi-level analyses for the used tests. 
Arrow T ’ represents the effect of the pretest on the posttest. Arrow ‘2’ represents the 
differential effect, operationalized by mathematical reasoning ability, which is a 
representation of the differential effect hypothesis. Arrow ‘3 ’ represents the remedial 
effect hypothesis, i.e., the effect of the experimental condition on the differential 
effect. Dotted arrows indicate the main effect, which it is necessary to control for. 
Three analyses were carried out, given that we administered three tests. 

1 . Mathematical reasoning ability test 

2. Domain specific test: Measurements and Measures 

3. Domain specific test: Information Processing 



Insert figure 3 here 



Table 4 shows the three starting models are presented of the three different tests used in this 
experiment. It shows the decomposition of the total variance of the posttest into a within-class 
part and a between-class part, indicated by a 2 and a 2 0 respectively. 

The within-class variance of the Mathematical Reasoning Ability test is 52.86 or 97 
percent and the between-class variance is 1 .82 or 3 percent. 

The within-class variance of the Measurement and Measure test is 82.95 or 82 percent 
and the between-class variance isl8.39 or 18 percent. 

The within-class variance of the Information Processing test is 35.6 or 89 percent and the 
between-class variance is 4.22 or 1 1 percent. 

All test have a small proportion between-class variance, which indicates that classes are 
unstreamed to a large extent. 



Insert table 4 here 
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Table 5 represents the outcomes of the multi-level analysis of the mathematical reasoning 
ability test. There is a differential effect (0.006), indicating that the stronger the students are, 
the more they gain in general. This ties in with the differential effect hypothesis. There is a 
positive interaction of the differential effect and the experimental condition, with a coefficient 
of 0.0005). This means that the stronger the students are, the more they profit from the 
experimental condition, which does not follow from the remedial effect hypothesis. It was 
expected that the low achieving students would profit from the remedial instruction given to 
them during the stage 4a en 4b. 



Insert table 5 here 



Figure 4 represent the outcomes of the multilevel analysis in Table 5, model 2. 



Insert figure 4 here 



There is no effect of the experimental program, these means that the students of both 
programs do not differ on the mean score. However there is a program effect on the 
differential effect in favor of the high-achieving students. 

Table 6 represents the results of the multi-level analysis of the Measurement and Measures 
test. There is no differential effect nor a remedial effect. 



Insert table 6 here 



Figure 5 represents the outcomes of the multilevel analysis Table 6 model 2. There is neither 
a program or differential effect. The pre-knowledge of the student has considerable influence 
on the posttest, while there seems no room for effects of the intervention program. Comparing 
the variance of the decomposition of variance and model 2 in Table 6 we observe a decline of 
30% of the variance on the student level. 



Insert figure 5 here 



The results of the multi-level analysis of the Information Processing test is represented in 
Table 7. 



Insert table 7 here 



In model 2 the effect of the pretest and the differential effect are introduced. In addition, the 
experimental condition is incorporated into the model. It turns out that the differential effect 
is negative (-0.002), which means that, in addition to the general effect of the pretest of 
Information Processing (its regression coefficient is 0.58), the stronger students tend to gain 
less than the weaker students. There is also a negative effect of the experimental condition on 
the differential effect; the coefficient being -0.002. This means that the stronger students are 
the less they profit from the experimental condition. In other words: the experimental 
condition enforces the advantage of the weaker students. The outcomes of the multilevel 
analysis in Table 7, model 2, are represented in the diagram in Figure 6. 



Insert figure 6 here 



Figure 6 shows that there is no direct program effect, meaning that the mean of the students in 
both programs do not differ. However, there is a negative program effect on the differential 
effect. This outcome means that the stronger the students are the lesser they benefit from the 
experimental program (or the other way around: the differential effect is mitigated by the 
program which indicates a compensating effect for the low-achieving students. 



Conclusions, discussion 

The central research question in this paper is: what are the effects of cooperative groups 

training in the use of both cognitive and social strategies on the learning results of high- and 

low-achieving students in secondary mathematics education? 

Three hypotheses were formulated. 

I. The program hypothesis'. The experimental program has a positive effect on learning 
outcomes 

II. The differential effect hypothesis'. High- and low-achieving students will benefit 
differently from cooperative learning. 

III. The remedial instruction hypothesis'. Low-achievers in the experimental program will 
benefit more from working in small groups than low achievers in the control program 

In the analyses of variance the program hypothesis is confirmed on two of the three measures. 
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Students in the experimental program gain significantly more than students in the control 
program on Mathematical Reasoning Ability and on a domain-specific test in mathematics 
(topic Measurements and Measures). The difference on the other domain-specific test ( 
Information Processing) is not significant. We now turn to the results of the multilevel 
analyses. In the multilevel analysis no program effect is found, neither on the slope nor on 
the intercept. 

In the multilevel-analysis, the differential effect hypothesis is partially confirmed. On two of 
the three criterion tests there is a differential effect. On the Mathematical Reasoning Ability 
test, high-achieving students gain more per score unit on the pretest than the low-achieving 
students. Here we see the differential effect hypothesis confirmed. No differential effect is 
found on the domain-specific test on Measurements and Measures: our hypothesis is not 
confirmed. The differential effect on the Information Processing test is negative: low- 
achieving students gain more per score unit than the high achievers. This last finding 
contradicts our hypothesis. 

In the multilevel-analysis, the remedial instruction hypothesis is confirmed only on one of 
the three measures. Low-achieving students in the experimental program gain more on the 
domain-specific test in Information Processing than the low-achieving students in the control 
program. This is in line with our hypothesis. However, no remedial effect is found on the 
domain- specific test in Measurement. Finally, the opposite is found in regard to 
Mathematical Reasoning Ability: high achievers in the experimental program gain more than 
the high achievers in the control program. This last finding contradicts our hypothesis. 

The analyses lead to conclusions in different directions. As far as Mathematical Reasoning 
Ability is concerned, the outcomes of this study shows that teaching students how to use 
problem solving strategies and to work effectively in small cooperative groups seem to be 
beneficial for the high-achieving students. This is in keeping with the research on strategy 
training. From the literature it is known that low-achieving students do not always benefit 
from this kind of intervention (Hattie, Biggs and Purdie, 1996). The high-achieving students 
are able to use problem solving strategies on a higher cognitive level. 

However, a different conclusion emerges about Information Processing, which shows a 
positive effect for the low- achieving students. This is in line with our Remedial Instruction 
hypothesis. The special instruction, guidance by the teacher and the tasks embedded in rich 
contexts might be the cause of this positive effect for the low-achieving students. 

No differential effects nor any remedial effects were found on the domain-specific subject 
Measurement and Meausures. Here the pretest has a high predictive value (.69) relative to the 
score on the domain specific posttest. In this series of lessons the students learned all kinds of 
calculation procedures. These procedures are rather straightforward. It seems that the pre- 
knowledge of this subject does not leave any room for intervention effects. 
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All in all, the results of this study only partially confirm our hypothesis. Nevertheless, the 
study shows some intriguing outcomes, which demand further conceptual analysis and 
empirical research. Some findings, especially those concerning the differential and remedial 
effects (for low- and high-achieving students) indicate that these kinds of effects are less 
stable over different measures and different studies. The most plausible explanation for the 
differences in outcomes on the standardized Mathematical Reasoning Ability test (in favor of 
the high achieving students) and the outcomes on the domain specific ‘ Information 
Processing’ test (in favor of the low-achieving students) test may lie in the content of these 
tests. Mathematical Reasoning Ability refers to a more stable ability (trait) and concerns ‘far 
transfer’. The domain specific test has a more direct relation to the mathematical content of 
the program (‘near transfer’). The remedial instruction (scaffolding) was given in relation to 
the specific mathematical content. As a result, low-achieving students in the experimental 
condition gain more on the domain specific test than low-achieving students in the control 
program. It would seem worthwhile to continue this line of research and conduct further 
studies into differential effects on both standardised ability tests and domain specific 
measures. 
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Figures to insert in the text 




Figure 1: Representation of the knowledge used during problem solving 
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Figure 2: Research design of the field experiment. 
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Figure 4: The outcomes of the multi-level model of the Mathematical Reasoning Ability 
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Figure 5: The outcomes of the multilevel analysis of the Measurement and Measures test 




Figure 6: The outcome of the multilevel analysis of the Information Processing test 
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Figure 3: The basic multi-level model 




Figure 4: The outcomes of the multi-level model of the Mathematical Reasoning Ability 
(T able 5 model 2). 




Figure 5: The outcomes of the multilevel analysis of the Measurement and Measures test 




Figure 6: The outcome of the multilevel analysis of the Information Processing test 
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Table 1: The reliability of the 


test used in 


this experiment 


Test 


Pretest 


Posttest 


Measurements and Measures 


0.78 


0.83 


Information Processing 


0.76 


0.60 



Table 2: The results on the pre-tests and post-tests (between parentheses the standard 
deviation), N -students =444, N -classes = 18. 





Control Program 


Experimental Program 




pre-test 


post-test 


pre-test 


post-test 


Measurements and Measures* 


15.16(8.01) 


30.29(10.26) 


18.45 (7.69) 


34.28 (10.45) 


Information Processing* 


18.45 (5.89) 


26.64 (6.83) 


19.95 (6.18) 


26.62 (6.02) 


Math. Reasoning ability 


51.56(8.66) 


51.96 (7.02) 


51.65 (9.24) 


53.50 (8.14) 



*The obtainable score on the pre-test and post-test differs 



Table 3: Characteristics of the regression and the explained variance(R^) of the post-test on 
the pre-test (the standard deviation between parentheses, N -students = 444, 14- 
classes = 18). 







Control Program 






Experimental Program 




a (intercept) b (slope) 


R 2 


a (intercept) b (slope) 


R 2 


Mathematical 
Reasoning Ability 


24.1 


0.54 (0.04) 


0.45 


27.7 


0.50 (0.05) 


0.33 


Measurement and 
Measures 


19.1 


0.71 (0.07) 


0.32 


20.1 


0.78 (0.08) 


0.33 


Information 

Processing 


16.0 


0.59 (0.07) 


0.27 


21.2 


0.27 (0.06) 


0.08 
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Table 4: Decomposition of the total variance of the test into a within-class part and a 
between-class part of the sub-scales 





Mathematical 
Reasoning Ability 


Measurement and 
Measures-2 


Information 

Processing-2 


Random Part 








a 2 Student 


52.86 (3.62) 


82.95 (5.69) 


35.6 (2.44) 


Co Intercept 


1.82 (1.33) 


18.39 (7.23) 


4.22(1.88) 


Likelihood 


3032.74 


3255.58 


2870.96 
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Table 5: Results of the multilevel analysis, in which the score on the Mathematical Reasoning 
Ability posttest is the dependent variable (standard deviation between parenthesis), 
N-students = 444, N-classes = 18. 



Mathematical Reasoning Ability 


Model 2 


Fixed part 




Mathematical Reasoning Ability- 1 


-0.12(0.20) 


Differential effect 


0.006 (0.002) 


Explanation of the between class slope 
differences differential effect by 




Experimental condition 


0.005 (0.0002) 


Random part 




a 2 Student 


33.54 (2.28) 


class: 




CTq Intercept 


0.0 (0.0) 


CT 01 


0.0 (0.0) 


ct 2 Differential effect 


4.7*10' 8 (l.OxlO' 8 ) 


Model statistics 




Likelihood ratio 


2822.01 
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Table 6: Results of the multilevel analysis, in which the score on the Measurement and 

Measures is the dependent variable (standard deviation between parenthesis), N- 
students = 444, N-classes = 18. 



Measurement and Measures 


Model 2 


Fixed part 




Pretest Measurement and Measures 


0.69 (0.06) 


Random Part 




ct 2 Student 


56.8 (4.0) 


class: 




CTq Intercept 


31.9(15.4) 


CT 01 


-0.8 (0.6) 


cr 2 Pretest Measurement and Measures 


0.03 (0.02) 


Model statistics 




Likelihood 


3092.03 
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Table 7: Results of the multilevel analysis, in which the score on the Information Processing 
posttest is the dependent variable (standard deviation between parenthesis), N- 
students-444, N-classes-18. 



Information Processing 


Model 2 


Fixed Part 




Pretest Information Processing 


0.58 (0.09) 


Differential effect 


-0.002 (0.001) 


Explanation of the between class slope 
differences differential effect by 




Experimental condition 


-0.002 (0.0006) 


Random Part 




a 2 Student 


28.0 (2.0) 


class: 




(Jo Intercept 


22.8(11.1) 


CT 01 


-0.01 (0.008) 


a 2 Pretest Information Processing 


7.3x1 O' 6 (5.7X10- 6 ) 


Model statistics 




Likelihood Ratio 


2770.23 
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